With the rapid development of machine learning technologies, model prediction has become an important online service. To provide model prediction services, corresponding models need to be loaded into a cluster node in advance.
In some existing systems, only one model service framework is started on a cluster node, and several models are loaded through the model service framework. However, when an exception occurs to the model service framework, the cluster node on which the model service framework is located needs to be restarted. After the cluster node is restarted successfully, the model service framework is restarted, and the models that are already deployed on the cluster node are reloaded.